A Signal for a Theory of Flavor at the Large Hadron Collider 
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In Randall- Sundrum models with gauge bosons and fermions in the extra dimensional bulk, it 
is possible to build models of flavor by localizing the fermions in the extra dimension. Since the 
Higgs must be localized at or close to the TeV scale fixed point, heavier fermions must be localized 
close to this brane. The first Kaluza-Klein excitations of the gauge bosons are also TeV-localized, 
so they have stronger couplings to heavier fermions leading to tree-level flavor-violating couplings. 
We investigate the potential of the LHC to observe flavor violation in single top production at very 
high invariant masses, in addition to the observation of the corresponding top-anti-top resonance. 
We conclude that the LHC will be able to observe tree-level flavor violation in single top production, 
probing KK masses at least as large as 2 TeV, as well as a very interesting region of the parameters. 

PACS numbers: ll.10.Kk, 12.60.-i, 13.90.+i 



Introduction It is generally believed that extensions 
of the standard model must address the stability of the 
weak scale with new physics at energies not far above the 
TeV scale. The origin of fermion masses, on the other 
hand, could in principle reside at much higher scales. 
Nonetheless, it is tempting to consider the possibility 
that the origin of at least some of the fermion masses is re- 
lated to TeV scale physics. This is particularly true when 
considering the origin of the top quark mass, which is of 
the order of the weak scale. The correspondingly large 
Yukawa coupling suggests the presence of a strongly cou- 
pled sector associated with the third generation quarks. 

Recently, Randall and Sundrum proposed the use of 
a non-factorizable geometry in five dimensions [l[ as a 
solution of the hierarchy problem. The extra dimen- 
sion is compactified on an orbifold S\jZ<i of radius r so 
that the bulk is a slice of AdSs space between two four- 
dimensional boundaries. The metric depends on the five 
dimensional coordinate y and is given by [l[ 



ds 2 



-^V^dx^dx" - dy 2 



(1) 



where x 11 are the four dimensional coordinates, o~(y) = 
k\y\, with k ~ Mp characterizing the curvature scale. 
This metric generates two effective scales: Mp and 
Mpe~ k7rr . In this way, values of r not much larger than 



(11 — 12)) can be used in order 
M P e- knr ~ O(TeV) on one of 



the Planck length (kr 
to generate a scale A r 
the boundaries. 

In the original RS scenario, only gravity was allowed 
to propagate in the bulk, with the Standard Model (SM) 
fields confined to one of the boundaries. The inclusion of 
matter and gauge fields in the bulk has been extensively 
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treated in the literature 0,0,11, SB- The Higgs field 
must be localized on or around the TeV brane in order to 
generate the weak scale. As it was recognized in Ref. 
it is possible to generate the fermion mass hierarchy from 
0(1) flavor breaking in the bulk masses of fermions. Since 
bulk fermion masses result in the localization of fermion 
zero-modes, lighter fermions should be localized toward 
the Planck brane, where their wave- functions have an 
exponentially suppressed overlap with the TeV-localized 
Higgs, whereas fermions with order one Yukawa couplings 
should be localized toward the TeV brane. 

Since the lightest KK excitations of gauge bosons are 
localized toward the TeV brane, they tend to be strongly 
coupled to zero-mode fermions localized there. Thus, 
the flavor-breaking fermion localization leads to flavor- 
violating interactions of the KK gauge bosons, partic- 
ularly with third generation quarks. For instance, the 
first KK excitation of the gluon, will have flavor- violating 
neutral couplings such as G^(t^ fJ 'T a q), where q = u,c. 
Consequences of such effects have been considered in 
Ref. [13]. Here, we will study the potential of the LHC 
to directly observe the flavor- violating couplings through 
the anomalous production of single top quarks at very 
high invariant masses. 

The action for fermion fields in the bulk contains a 
bulk mass term as in [H, 0| 



Sf = 



J d 4 x dy V?{ sgn(y)M)**} 



(2) 



where the dots stand for the fermion kinetic terms. The 
bulk mass term Mf in eqn. is expected to be of order 
k ~ Mp. Although the fermion field \& is non-chiral, we 
can still define ^t,R = |(1 T Ts)*- The KK decomposi- 
tion can be written as 



* / )tt<v 1 ■ 



(3) 



n=0 



where tp^' R (x) corresponds to the nth KK fermion ex- 
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citation and is a chiral four-dimensional field. The zero 
mode wave functions are 



f R ' L (v) 



' 2fc7rr(f ±2cr,l) ± CR: 

ofe7rr(l±2 Cii . L ) _ I e 



k y 



(4) 



with CR t L = Mf/k parametrizing the bulk fermion mass 
in units of the inverse AdS radius fc, and naturally of 
0(1). The Z2 orbifold projection is used so that only 
one of these is actually allowed, either a left-handed or a 
right-handed zero mode. 

Fermion bulk mass parameters c l L and c R are con- 
strained by the requirement that we obtain the correct 
fermion masses and mixings with 5D Yukawa couplings 
of order one. For instance, if for simplicity we assumed 
that the Higgs is localized on the TeV brane, the Yukawa 
couplings of bulk fermions can be written as 



^i(x,y)S(y-' ! rr)H(x)^ j (x,y) , 

(5) 

where Afj° is a dimensionless parameter and M5 is the 
fundamental scale or cutoff of the theory. In order for 
this model to be a natural solution to the fermion mass 
hierarchy problem, we need |Afj°| ~ 0(1). The resulting 
zero-mode fermion mass matrices have then the form 




Mi. 



X 5D v 



(0) 



where v ~ 174 GeV is the Higgs VEV. Thus we can see 
that in order to get small masses, light fermions should 
be localized towards the Planck brane. This results in 
their couplings to KK gauge bosons being mostly univer- 
sal leading to no observable FCNC effects. On the other 
hand, third-generation quarks are required to be close to 
the TeV brane so as to generate a large enough top mass. 
This posses a potential problem: both the right handed 
top quark and the left-handed doublet Q 3 , should be sig- 
nificantly localized towards the "Higgs" brane, meaning 
that they can have rather strong couplings to the first 
KK excitations of gauge bosons. This induced flavor vi- 
olation of KK gauge bosons with b^ (we assume bn lo- 
calized on the Planck brane) is, in principle, constrained 
by the precise measurement of the Z — > bb interactions 
at the Z-pole. Depending on the model, this leads to a 
bound on c 3 as well as to a typical KK gauge boson 
mass bound of ^ 3 TeV. Also electroweak precision con- 
straints require that the gauge symmetry in the bulk be 
enlarged to SU(3) c x SU(2) L x SU(2) r xU(1)b-l- More 
recently, and in the context of these very same models, it 
has been argued that it is possible to protect the Z — > bb 
coupling from large corrections by new symmetries [Io| . 
resulting in potentially lower KK masses. Also, in mod- 
els without a Higgs 11] or with a bulk Higgs [12], a KK 
scale as low as 1 TeV can be readily accommodated. For 
the purpose of this work, we will be only concerned with 



the strong interactions, and not with the details of the 
electroweak sector. Thus, to be general enough, we will 
consider KK gauge boson masses as low as 1 TeV. 

We will consider the range of parameters defined by 
c\ = [0.3,0.4] and c R = [-0.4,0.1]. This is chosen such 
that although the correct top mass is obtained, no con- 
tradictions with electroweak measurements are present, 
particularly with the Z — ► bb rate. We will only take 
the combinations of parameters in these ranges that sat- 
isfy the requirement imposed by the quark masses and 
mixings. 

Couplings We will assume that the only non-universal 
couplings are those of the t R , and b^ with the KK 
gauge bosons. All other fermions, including the right- 
handed b quark are assumed to be Planck brane local- 
ized. The non-universality of the KK gauge boson cou- 
plings leads to tree-level flavor violation. The diagonal- 
ization of the quark mass matrix requires a change of 
basis for the quarks fields. In the SM, this rotation leads 
to the CKM matrix in the charged current, but the uni- 
versality of the gauge interactions results in the Glashow- 
Ilyopoulos-Maiani (GIM) mechanism in the neutral cur- 
rents. However, since the KK excitations of the gauge 
bosons are non-universal, tree-level GIM-violating cou- 
plings will appear in the physical quark basis. 

We will consider the dominant non-universal effect as 
coming from the couplings of ie, tt and b^ to the first 
KK excitation of the gluon: g tR , gt L and gb L respectively. 
The SU(2)l bulk symmetry implies gt L = gb L - In order 
to obtain these effective 4D couplings we need to compute 

g 5 J d A x J dy^g-y(x,y)^T a G«(x,y)y(x,y) (7) 

where g$ is the 5D SU(3) C coupling, and the KK ex- 
pansion of the gluon G°(x,y) and its wave- functions 
in the extra dimension can be found in The cou- 
plings of eqn. |(7J| depend on the fermion bulk masses. 
For the range given above for c\ and c R we obtain 
9tL = 9b L = [1.0, 2.8] g s and g tR = [1.5, 5] g s , where g s 
is the usual 4D SU(3) C coupling. The universally cou- 
pled light quarks as well as the right-handed b quark 
have g\= g q R = g b R ~ -0.2 g s . 

We are now in a position to compute the effects of all 
the flavor- conserving interactions of the (x), the 
first KK excitation of the gluon. We can also compute 
the width of the KK gluon. This can be written as 



12 



M G (9g, 



•2& 



9t RJ 



(8) 



where we defined the g 9 's as the couplings in units of 
the strong coupling <? s , and we neglected the flavor- 
violating contributions. Thus, the range of couplings de- 
fined above will result in a minimum and a maximum 
width for the KK gluon. These are r m ; n . ~ 0.04 Mq and 
r max . ~ 0.35 Mq. Then, we see that the range of values 
for the couplings allow for rather narrow or rather broad 
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resonances, two very different scenarios from the point of 
view of the phenomenology. 

We can also estimate now the size of the fla- 
vor changing couplings. These arise as a conse- 
quence of the rotation to the quark mass eigen- 
basis. Then, if the CKM matrix is generated cor- 
rectly, in this physical basis the left-handed up-type 
quarks couple to the KK gluon through the fol- 
lowing currents: Ug {IlT^^l), U{ c (i L T a ^c L ) and 
U L U (itT'^Ui). Similarly, the right-handed up- type 
quarks couple through U t R t (t R T a lfi t R ), 11% (t R T a ltl c R ) 
and 11% (t R T a 'f fi u R ). Here, Ul and U R are the left- 
handed and right-handed up-type quark rotation matri- 
ces responsible for the diagonalization of the Yukawa cou- 
plings of the up- type quarks. These are not observable 
in the SM, but here they govern the flavor-violating am- 
plitudes. In principle, we have little knowledge of the 
entries of these matrices since there are many possible 
choices for textures that would result in the correct quark 
masses and mixings. However, given that we know that 
Vckm = u\Dl, we can assume that Ul — yfVcKM , and 
similarly for D L . This leads to U% ~ V c b ~ 0.04 and 
U L U ~ V u b — 0.004. On the other hand, we have no bias 
from the SM on the entries of U R . Constraints on its 
elements can be imposed by asking for the correct up- 
quark masses after diagonalization. However, values of 
U% ~ 0(1) are still possible. For instance, if most of the 
charm quark mass comes from the mixing with the third 
generation up quark, then U% = 1 is possible [14]. This 
is even more so if Alf < A|f , while both still of 0(1). In 
what follows we will consider U% and U% as free param- 
eters. Since we will assume no separation of charm from 
light jets, we define = y/Tuff + (U L U ) 2 , and study 
the sensitivity of the LHC to this parameter for a given 
KK gluon mass. 

Signal and Backgrounds These flavor-violating inter- 
actions could be directly observed by the s-channel pro- 
duction of the first KK excitation of the gluon with its 
subsequent decay to a top and a charm or up quark. For 
instance, at the LHC we could have the reaction 

pp -» G«« -» tq , (9) 

with q = u, c. Thus, the Randall-Sundrum scenario with 
bulk matter predicts anomalous single top production at 
a very high invariant mass, which is determined by the 
mass of the KK gluon. In what follows, we study in detail 
the potential of the LHC for observing this signal, with 
a top quark and a jet with very high invariant mass. 

We will present results for the sets of values for c\ and 
c l R that result in the minimum width for the KK gluon, 
as estimated above. 

In order to reduce the backgrounds we considered 
only the semi-leptonic decays of the top quarks pp — > 
tq (tq) — > b£ + ueq {bl~T>tq), where I = e or n, and q = u,c. 
Therefore, this signal exhibits one b jet, one light jet, a 



charged lepton and missing transverse energy. There are 
many SM backgrounds for this process, namely: 

• pp — ► tt — > b£ + veb£~9e when one of the b jets is 
mistagged and one of the charged leptons is either 
lost in the beam pipe or embedded in one of the jets. 
It is interesting to notice that the flavor-conserving 
signal for the first KK state of the gluon decaying 
to tt pairs also contributes to this background. 

• pp — > W ± jj — > i vjj where one of the light jets is 
tagged as a b jet. 

• pp — > W^bb — > l^vbb where one of the b jets is 
mistagged. 

• pp — > W*^ — > tb + ib — » bbl^v where one of the b 
jet is mistagged. 

• pp — > tbj — > bbjt^v where one of the jets is lost 
and just one jet is tagged as a b jet. Here in the 
M^-gluon fusion we considered only the diagrams 
that are not higher order corrections to the single 
production of tops via W* . 

We simulate the signal and backgrounds with the package 
MADEVENT We initially imposed the following jet 
and lepton acceptance cuts 

p> T > 20 GeV , |%| < 2.5 , 

p l T > 20 GeV , \y e \ < 2.5 (10) 

AR tj > 0.63 , AR U > 0.63 , 

where j can be either a light or a b jet. 

In order to further reduce the background we imposed 
the following additional cuts 

1. We require that the invariant mass of the system 
formed by the lepton, the b tagged jet and the light 
jet be within a window 

M G(1) - A < M m < M G( i) + A (11) 

around the first KK excitation of the gluon mass. 
This cut ensures that the selected events have large 
invariant masses, as required by the large mass of 
the s-channel object being exchanged. The values 
of A used in this study are presented in Table |TJ 

2. We require the transverse momentum of the light 
jet to be larger than p cu t, i.e., 

Pj light > Pcut (12) 

Since the light jet in the signal recoils against the 
top forming with it a large invariant mass, it tends 
to be harder than the jets occurring in the back- 
ground. We present in Table |T] the values for p cut 
used in our analysis. 
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M am (TcV) 


A (GeV) 


Pcut (GeV) 


1 


120 


350 


2 


250 


650 



TABLE I: Cuts used in our analysis. See text for details. 



3. We also require that the invariant mass of the 
charged lepton and the b tagged jet be smaller than 
250 GeV: 



M M < 250 GeV 



(13) 



This requirement is always passed by the signal, 
but eliminates a sizable fraction of the Wjj back- 
ground. It substitutes for the full top reconstruc- 
tion when the neutrino momentum is inferred, 
which we are not using here. 

We present in Table [TT] the cross sections for signal 
and backgrounds for M G (i) = 1 TeV. The main sources 
of backgrounds are Wjj and ti production. The latter 
includes the KK gluon flavor conserving signal. Thus, 
this background partly scales with Mg the same way the 
signal does. The signal is obtained for U^f = 1 and ne- 
glecting the contributions from left-handed final states, 
corresponding to U£ = 0. The latter approximation is 
justified since we assume the left-handed quark rotations 
to be proportional to the CKM matrix, a conservative 
assumption. Regarding the choice of bulk masses, we 
fix these so as to obtain the minimum width which, as 
mentioned above, can be as small as Tq ~ 0.04Mg. Ta- 
ble [nT] shows the results for Mq = 2 TeV under similar 
assumptions. 

In order to evaluate the reach of the LHC, we will 
require a significance of 5 a for the signal over the back- 
ground. For a given KK gluon mass and accumulated lu- 
minosity, this can be translated into a reach in the flavor- 
violating parameter U^f defined above. This is shown in 
Table IIVI We see that the LHC will be sensitive to tree- 
level flavor violation for KK gluon masses of up to at 
least 2 TeV, probing a very interesting region of values 
for Uft. The reach can be somewhat better if we allow 
for the reconstruction of the momentum of the neutrino 
coming from the W decay, which typically reduces the 
Wjj background more drastically. 

Conclusions We have performed the first study of 
flavor violation at the LHC mediated by a high invari- 
ant mass object. Unlike rare top decays or low energy 
processes, this signal can only be a consequence of the 
tree-level flavor violation characteristic in these models. 
These phenomenon is expected in theories with one ex- 
tra dimension with Anti-de Sitter curvature in the bulk, 
where gauge and matter fields are allowed to propagate 
in the bulk, and is an essential manifestation of a the- 
ory of flavor. Its observation would constitute a pri- 



Process 


a - JEE]) 


a - CE2) 


a - (p)) 


PP -> t j 


148 fb 


103 fb 


103 fb 


pp Wjj 


243 fb 


42.0 fb 


21.0 fb 


pp — > Wbb 


11.1 fb 


4.07 fb 


3.19 fb 


pp — > tb 


1.53 fb 


0.70 fb 


0.61 fb 


pp — > ti 


44.4 fb 


15.1 fb 


14.2 fb 


Wg fusion 


32.0 fb 


5.23 fb 


5.23 fb 



TABLE II: Signal and background cross sections for a KK 
gluon of M G (i) = 1 TeV, after the successive application of the 
cuts denned in (|lip . (|12[) and (|13[) . Efficiencies and b tagging 
probabilities are already included. Here we used = 1. 



Process 


a - CP 


a - CCl 


o - (O 


PP ->• tj 


5.10 fb 


2.18 fb 


2.18 fb 


pp -> Wjj 


25.4 fb 


3.79 fb 


0.95 fb 


pp — > Wbb 


0.97 fb 


0.45 fb 


0.06 fb 


pp — > tb 


0.04 fb 


0.02 fb 


0.02 fb 


pp — > ti 


1.60 fb 


0.29 fb 


0.24 fb 


Wg fusion 


1.20 fb 


0.10 fb 


0.10 fb 



TABLE III: Signal and background cross sections for a KK 
gluon of M G (i) — 2 TeV. Efficiencies and b tagging probabil- 
ities are already included. Here we used U% = 1 



mary test of this aspect of bulk Randall-Sundrum mod- 
els. We conclude that the LHC has the potential to ob- 
serve this phenomenon up to KK gluon masses of at least 
Mq(i) — 2 TeV for interesting values of the parameter 
U% (see Table HVj). In this study, we used narrow res- 
onances. A more thorough study of the LHC reach for 
broader resonances is needed. Among other aspects for 
further study are the impact on the background of using 
a full top reconstruction, and of radiative corrections and 
the choice of renormalization scale. We leave these more 
detailed studies for future work [TI ]. 
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M G [TeV] 


30/6- 1 


100/&- 1 


300/6- 1 


1 


0.24 


0.18 


0.14 


2 


0.65 


0.50 


0.36 



TABLE IV: Reach in Ufl for various integrated luminosities. 
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